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A  Generalization  of  Autocorrelation  and 
Partial  Autocorrelation  Functions  Useful  for 
Identification  of  ARMA(p,q)  Processes 

1.  Introduction. 

For  identification  of  the  order  of  moving  average  process  or  autoregressive 
process  the  autocorrelation  function  p^  and  the  partial  autocorrelation  function 
p'^  have  been  successfully  used  (see  Box  and  Jenkins,  1976).  For  general  mixed 
ARMA(p,q)  processes  neither  the  autocorrelation  function  pk  nor  the  partial 
autocorrelation  function  p£  show  a  simple  behavior  which  leads  to  a  simple  and 
useful  procedure  of  determining  the  orders  p  and  q. 

In  this  article  we  propose  a  new  definition  of  generalized  autocorrelation 
function  p (k  ,&)  (k  >  0,  a  >  0)  with  the  following  properties. 

(PI )  p (1  >£)  =  p^,  £  ^  1 , 

(P2)  p (k,l )  =  p',  k  >  1, 

(P3)  -l<p(k,£)<l, 

(P4)  For  an  ARMA(p,q)  process 

p(k,£)  =0  if  k  >  p  and  i  >  q. 

The  first  property  (PI)  says  that  when  p(k,£)'s  are  arranged  in  a  two  dimensional 
array,  then  the  first  row  coincides  with  autocorrelation  coefficients.  Similarly 
(P2)  says  that  the  first  column  coincides  with  partial  autocorrelation  coefficients. 
p(k,£)  is  in  fact  a  correlation  coefficient  between  certain  random  variables 
and  this  is  reflected  in  (P3).  (P4)  is  the  main  property  of  the  proposed 

procedure  that  makes  it  useful  for  the  identification  of  ARMA(p,q)  processes. 
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Recently  several  generalizations  of  autocorrelation  and  partial  autocorrelation 
functions  have  been  proposed  for  the  purpose  of  identification  of  ARMA(p,q) 
processes.  They  include  R-  and  S-arrays  of  Gray,  Kelley,  and  Mclntire  (1978); 
"GPACF"  (Generalized  Partial  Autocorrelation  Function)  discussed  in  Woodward 
and  Gray  (1981)  Tiao  and  Box  (1981),  and  Jenkins  and  Alavi  (1981);  Corner  Method 
of  Beguin,  Gourieroux,  and  Monfort  (1980);  ESACF  (Extended  Sample  Autocorrelation 
Function)  of  Tsay  and  Tiao  (1984);  SCAN  (Smallest  CANonical  correlation)  of  Tsay 
and  Tiao  (1983).  The  basic  ideas  underlying  these  procedures  are  quite  similar. 

Some  differences  between  these  procedures  are  discussed  in  Tsay  and  Tiao  (1983)  and 
Tsay  and  Tiao  (1984).  ESACF  and  SCAN  of  Tsay  and  Tiao  seem  to  be  more  sophisticated 
among  these  procedures.  The  major  advantage  of  ESACF  and  SCAN  is  that  they  can 
handle  nonstationarity  (unit  roots)  in  autoregressive  part  directly.  The 
procedure  proposed  below  is  restricted  to  stationary  ARMA  processes.  On  the 
other  hand  the  proposed  procedure  has  the  following  attractive  properties: 
a)  it  contains  autocorrelation  and  partial  autocorrelation  coefficients  as  - 
special  cases,  b)  simple  asymptotic  theory  generalizing  the  usual  asymptotic 
theory  for  autocorrelation  and  partial  autocorrelation  coefficients,  c)  quick 
computation  by  recursive  formulae  generalizing  the  Durbin-Levinson  recursive 
method  (Durbin  (1960),  Levinson  (1946)).  Because  of  these  properties,  the 
proposed  procedure  seems  to  be  a  natural  generalization  of  autocorrelation 
and  partial  autocorrelation  functions.  It  is  also  an  elaboration  of  ideas  in 
Section  5  of  Bartlett  and  Diananda  (1950). 

In  Section  2  the  definition  of  the  generalized  autocoreel ati on  function 
is  given  and  then  the  properties  (PI)  -  (P4)  are  proved.  In  Section  3  the 
asymptotic  distribution  of  the  sample  generalized  autocorrelation  function  is 
studied.  Theorem  3.2  unifies  and  generalizes  the  well  known  results  on  the 
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asymptotic  distributions  of  sample  autocorrelation  coefficients  r.  (k  >  q) 

for  MA(q)  processes  and  the  sample  partial  autocorrelation  coefficients  r,1 

k 

(k  >  p)  for  AR(p)  processes.  In  Section  4  recursive  formulae  for  computing 
p(k,£)  are  given.  This  recursive  procedure  is  straightforward  generalization 
of  the  well  known  recursive  procedure  of  obtaining  partial  autocorrelation 
coefficients  (Durbin  (1960),  Levinson  (1946)).  In  Section  5  some  simulation 
results  are  given  to  illustrate  the  behavior  of  the  proposed  procedure. 

Throughout  this  article  we  mean  by  ARMA ( p , q )  process  the  time  series  x. 

t 

generated  by 


i=0 


Vt- 


j=o 


a  .V, 

j  t- 


or 


e(L)xt  =  a(L)vt  5 

where  =  aQ  =  1,  L  is  the  lag  operator,  e(L)  =  1+6-jL+.  . .+3pLp,  a(L)  = 

1  +ot i L+ . .  .+otqL^ ,  and  vt  is  the  white  noise  term  with  Ev^  =  o^.  Furthermore 
for  regularity  we  assume  that  the  roots  of  the  polynomial  equation  3(L)  =  0 
lie  outside  the  unit  circle  and  vt's  are  independently  and  identically  distributed. 
We  will  not  repeat  these  assumptions  later. 

2.  Definition  and  Some  Properties  of  the  Generalized 
Autocorrelation  Function. 

Let  a(k)  denote  the  autocovariance  function  of  a  second-order  stationary 
sequence.  For  k  >  0  let  a(£,k)  =  (a(i),  a(£+l ) , . . .  ,a(£+k-l ) ) 1 ,  and  = 

(a(£) ,a(£-l ) , . . . ,a(£-k+l ) ) 1 .  Let  £(£,k)  denote  a  kxk  (nonsymmetric)  Toepl itz 
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matrix  with  a{i)  on  the  main  diagonal,  i .e. , 


1(4, k) 


(4) 

a(£+l)  ... 

a(£+k-l ) 

(4-1) 

e 

cr(4)  ... 

9 

a(£+k-2) 

e 

• 

U-k+1) 

O 

O 

a(£-k+2)  ... 

• 

a(4) 

(2.1) 


Now  we  give  the  definition  of  a  generalized  autocorrelation  function. 

Def i ni ti on .  Let  k  0,  £  _>  0.  A  generalized  autocorrelation  function  p(*,») 
is  defined  as 


p(k+l  ,5.+! )  =  [a(k+£+l )  -  a(£+l  ,k)  ’£(£,k)  ^a(k+4,-k)] 

/[a(0)-2a(l,k)'Z(£,k)'”1a;(£+l,k)  (2.2) 

r\j  f  u  'v 

+a(£+l  ,k)'z(£,k)"1^(0,k)z(jl5k),"1a(£+l,k)] 
if  |^(£,k) |  f  0, 

=0  if  |z(£,k) |  =  0. 

When  k  =  0  terms  involving  z(£,0)  are  taken  to  be  zero  since  they  are 
zero-dimensional.  Hence  p(l  ,£+l  )=o(£+l  )/a(0)=p£+-| .  It  is  also  easy  to  check 
that  when  £=0  then  p(k+l ,1 )  coincides  with  the  partial  autocorrelation  function 
P|^+1 .  The  properties  (PI),  (P2)  are  verified. 

We  now  discuss  the  motivation  for  the  above  definition  and  then  prove  the 
properties  (P3)  and  (P4) .  For  a  set  of  arbitrary  numbers  b,  ,...,b.  ,  (bg^l ) ,  let 
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k 

I  b-x.  .  =  b(L)x  , 
i=0  1  t_1  t 


(2.3) 


where  L  is  the  lag  operator  and  b(L)  =  1+b-jL+o . .+bkLk.  Let  P  =  L"1  be 
the  forward  shift  operator  and  let 


1  ”  Zt-£-k-l  b(p)xt- 


£-k-l 


k 

ll0  Vt-J-k-l+i' 


(2.4) 


Note  that  Z  is  defined  in  terms  of  the  reversed  process,  x  The  time 
reversibility  of  the  covariance  structure  of  stationary  sequence  is  essential 
in  considering  Z.  Now  consider  the  correlation  coefficient  between  Y  and  Z: 
Cor(Y,Z).  By  the  time  reversibility  of  the  process  we  have 


Var(Y)  =  Var(Z) 

■ftV  0’k+1)b 

*  <i(0)+2ft(i)«<1’k)H'fe(])J(0’k)fe(i)  (2-5) 

k 

■  l  b.b.a(i-j), 
i,j=0  1  3 

where  b  =  (1  ,b-| , . . .  ,b^) 1  and  b^  =  (b-j , . . .  ,b^) 1 .  Note  that  Var(Y)  can  be 
written  as  b(P)b(L)a(0)  where  P,  L  are  now  operating  on  the  argument  i  of  a(i). 
(See  the  notation  H^.  in  Bartlett  and  Diananda,  1950).  The  covariance  between 
Y  and  Z  is  given  by 


Cov(Y,Z)  =  b'zU+1  ,k+l)b 

=  ^(^+k+D+2bJ1){j(£+k,-k)+bj1)E(il,k)b(1)  (2.6) 

k 

=  l  bjb.ff(jt+k+l-1-j) 
i  >j=0  1  3 


-  b(L)Za(&+k+l ) , 
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where  b  =  (b^  ,bjc_-j . .  ,bg) 1  and  b^  =  (b^, . . .  ,b^ )' .  In  general  we  will  place 
a  ^  over  a  vector  to  denote  the  vector  with  its  elements  in  the  reversed  order, 
for  example,  a(t+k,-k)  =  a(£+l ,k) .  Combining  (2=5)  and  (2=6)  we  have  Cor(Y,Z)  = 
b(L)2a(k+Ji+l)/b(L)b(P)a(0). 

Now  we  choose  a  particular  set  of  (b-j , . . .  jb^) 1  =  fe(-j)  defined  by 


^(#.,k)  i  ^  — J^( 4^"1 , k ) ,  (2.7) 

or  b^  =  -z($,,k)  '~^a(£+l  ,k)  provided  that  z(&,k)  is  nonsingular.  With  this 
choice  of  b^-Q,  Cor(Y,Z)  reduces  to  the  right  hand  side  of  (2.2)  and  this 
proves  the  property  (P3) .  Note  that  (2.7)  can  also  be  expressed  as 

z(Jt,k)b(1)  =  -a(a+k,-k).  (2.8) 

It  remains  to  show  (P4).  Note  that  (2.7)  is  the  set  of  k  Yule-Walker 
equations  for  the  following  ARMA(k,£)  process: 

Yt  =  3(L)xt  =  a(L)vt, 

t/  n 

where  g(L)  =  1  +  g-|L+...+g^L  ,  a(L)  =  1  +  a,L+. . .+a£L  ,  and  v^'s  are  the  white 
noise  terms.  If  zU,k)  is  nonsingular  then  (2.7)  gives  the  values  of  the 
parameters  Now  the  smallest  time  index  in  a(L)vt  is  t  -  a  in  a£vt_£ 

and  the  largest  time  index  in  Zt_£_^_^  is  t  -  l  -  1  in  6|<xt_£_i .  Therefore 
Y  =  Y.  and  Z  =  Zt_£_^_^  are  uncorrelated  and  p(k+l  ,£+l )  =  0.  Note  that  if 
k  >_  p,  i  >_  q  then  an  ARMA(p,q)  process  is  a  special  case  of  ARMA(k,£)  processes 
having  3p+-j  =  . . .  =  3^  -  0  and  a  +-j  =  ...  =  a£  =  0.  This  proves  the  property. 
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(P4)  with  the  provision  that  p(k+l,A+l)  =  0  if  |s(£,k)|  =  0.  Now  all  properties 
(PI )-(P4)  have  been  verified. 

We  have  mathematically  verified  all  properties  (PI )-(P4) ,  but  the 
motivation  for  defining  p(k+l ,£+l)  =  0  when  |z(£,k)|  =  0  has  to  be  discussed 
more  carefully  for  further  developments  in  the  next  section.  An  important 
question  is  whether  l(£,k)  is  nonsingular  or  singular  when  k  _>  p,  i  >_  q  for 
an  ARMA(p ,q)  process.  For  an  nxn  matrix  A  let  N(A)  denote  the  null  space  of 
A  and  let  v(^)  be  the  nullity  of  namely 

v(A)  =  dim(N(A))  =  n  -  rank  (A). 

By  a  nondegenerate  ARMA(p,q)  process  we  mean  the  process  b(L)x.  =  a(L)v. , 
where  deg  g(L)  =  p,dega(L)  =  q,  3  *  0 5  and  ct  f  0. 

r  M 

Proposition  2.1.  Consider  a  nondegenerate  ARMA(p,q)  process.  If  k  >_  p, 

£  >_  q,  then 


v(S ( A >k) )  =  min(k-p,£-q) . 


(2.9) 


Furthermore  if  v  =  v($(£,k))  >  0,  then  a  basis  of  N(z(£,k)')  is  given  by 

$1  -  (Bq>^i » • • • »®p>0, . . . ,0)  ,  ^2  =  (0 ,3q » . . .  »3p ,0, . . .  ,0) 1 , . . . ,  =  (O,...,0, 

Sq> • • • »B  >0, . . . ,0) 1 ,  where  Bq  -  1  and  ^  has  first  v-1  zero  elements.  Similarly 
a  basis  of  N(z(£,k))  is  given  by  tj,  with  their  elements  in  the  reversed 

order. 

The  proof  of  this  proposition  will  be  given  in  Appendix.  Now  consider 
(2.7).  It  has  a  specific  solution  jf  =  (b,  , . . . ,3  ,0, . . . ,0) 1 .  A  general 

1  f-' 

solution  of  (2.7)  is  then  b(1)  =  (b1,...#bk)'  =  +  I^c.q.  where  c1,...,cv 
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are  arbitrary  constants.  With  these  b..  's  consider  Y  in  (2.3)  and  Z  in  (2.4). 

We  claim  that  Cov(Y,Z)  =  0  and  Var(Y)  =  Var(Z)  is  bounded  away  from  zero,  so 
that  Cor(Y,Z)  =  0.  By  Proposition  2.1  we  have  n(£,k)n.  -  n!s(£»k)  =  0, 
i  =  1 . .  Also  by  (2.8)  we  have  n  !cr(£+k,-k)  =  0,  i  =  1 , . . .  ,v.  Using  these 

*\j  I  'Xf 

results  in  (2.6)  we  obtain 

P 

Cov(Y,Z)  =  l  B-B-crU+k+l-i-j). 

1,4-0  1  3 

But  this  is  zero  by  the  same  argument  given  above  to  prove  the  property  (P4). 

Furthermore  Y  =  b1.xt_1.=B(L)xt+^_iC1.B(L)xt_.=a(L)vt+^_iC1.a(L)vt_^ .  Hence 

o 

Var(Y)  >_  av  for  all  c-j»...,c^.  This  proves  the  above  claim.  In  terms  of 
generalized  inverses  we  have  proved  the  following: 

Proposition  2.2.  For  an  ARMA(p,q)  process  the  first  expression  of  (2.2)  is 
zero  for  all  (k,£)  such  that  k  >_  p  and  £  >_  q,  if  z(£,k)"^  is  taken  as  any 
generalized  inverse. 

This  proposition  shows  that  it  is  logically  consistent  to  define  p (k+1 ,£+l )  =  0 
if  |z U,k) |  =  0.  We  will  see  in  the  next  section  that  these  considerations  are 
essential  for  the  discussion  of  sample  generalized  autocorrelation  coefficients. 

3.  Asymptotic  Distribution  of  Sample  Generalized 
Autocorrelation  Coefficients. 

In  this  section  we. discuss  distributional  properties  of  sample  generalized 
autocorrelation  coefficients.  In  practical  applications  sample  autocovariances, 

A 

a{i) ,  £  =  0,1,...,  are  computed  from  observed  time  series.  These  sample 
autocovariances  can  now  be  substituted  into  (2.2).  We  call  the  resulting  quantity 
sample  generalized  autocorrelation  coefficient  and  denote  it  by  r(k+l ,£+l ) .  These 
sample  generalized  autocorrelation  coefficients  can  be  arranged  in  a  two-dimensional 


9 


table.  If  there  is  a  pair  (p,q)  such  that  r(ks£)'s  are  close  to  zero  for  all 
pairs  (k,£)  such  that  k  >  p  and  £  >  q,  then  this  indicates  that  the  observed 
time  series  comes  from  an  ARMA(p,q)  process. 

Clearly  we  need  to  assess  the  sample  variability  of  r(k,£)'s  for  judging 
whether  they  are  significantly  different  from  zero  or  not.  Suppose  that  an 
observed  time  series  xt  is  a  nondegenerate  ARMA(p,q)  process.  The  following 
question  arises:  Does  r(k,£)  tend  to  be  small  for  (k,£)  where  k  >  p  and 

/S 

£  >  q?  The  difficulty  here  is  that  as  a  sample  quantity  the  matrix  z(£,k)  is 
nonsingular  with  probability  one  so  that  the  first  expression  of  (2.2)  is 
always  used  although  the  population  matrix  z(£sk)  is  singular.  Using  Proposition 
2.1  we  can  show  that  r(k,£)  tends  to  be  small  indeed. 

Theorem  3.1 .  Let  r(k+l,£+l)  be  the  sample  generalized  autocorrelation  coefficient 
obtained  from  an  ARMA(p,q)  process  of  length  T.  Then  for  k  p,  £  _>  q, 
r(k+l,£+l)  -  0p(T"1/2). 

Proof.  We  first  note  that  the  denominator  of  r(k+l ,£+l)  stays  bounded  away  from  zero 
in  probability.  This  can  be  shown  as  follows.  As  in  (2.5)  the  denominator  is 

A 

of  the  form  b's(0,k+l)b  where  the  first  element  of  b  is  equal  to  1.  Hence 
| |b| |  j>  1  and  this  implies  that  the  denominator  is  bounded  from  below  by  the 
smallest  characteristic  root  of  z(0,k+l ) .  However  the  smallest  characteristic 

A 

root  of  z(0,k+l)  converges  in  probability  to  the  smallest  characteristic  root 

of  z(0,k+l)  which  is  positive  because  £(0,k+l)  is  positive  definite.  This  shows 

that  the  denominator  is  bounded  away  from  zero  in  probability.  Therefore  it 

suffices  to  show  that  the  numerator  is  of  the  order  0  (T-^2).  Note  that 

P 

Num  =  cr(k+£+l  )-a(£+l  ,k)  'z(£,k)-1a(k+£,-k) 

=  (-1 )^det£(£+l ,k+l )/detz(£,k) . 
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Consider 

Pr{|Num|<cr1/2}=Pr{T(v+1)/2|de4(£+l5k+l)|<cTv/2|det^(£,k)|}, 
where  v  -  min(k-p,£-q) . 

Let  the  singular  value  decomposition  of  £($,+1  ,k+l )  and  l{z,k)  be 
z(5.+l  ,k+l )  =  Q-jDgg’ 


where  J^-j  >  ^2  are  or^°9ona^  matrices  and  {)  and  E,  are  diagonal  matrices 

with  diagonal  elements  d^.  =  6..,  i-1 , . . . ,k+l ,  and  e^.  =  e. ,  i=l ,. . . ,k, 
respectively.  Here  we  specify 


61  -  * *-6k-v  >  0  6k-v+l 


:<Sk=5k+l 


e,  > . . .  >£,  >  0  =  s,  ,  *» 

1  —  —  k-v  k-v+1 


"ek‘ 


Let  6  =  (d. .)  =  Q-!  z (£+1  ,k+l )QL  and  E  -  (e.  .)  =  R1  z(ji,k)RL.  From  the  joint 

1  J  Oj  I  ^  *Vr  1  J  'Xi  |  *u 

asymptotic  normality  of  sample  covariances  (see  Sections  8.3  and  8.4  of 

Anderson,  1971,  for  example)  it  follows  that  d.  .  =  d.  .  +  0  (T“^2) ,  e.  .  =  e .  .+0  (T“^2), 

'  J  *  J  r  '  vl  'Jr 

hence 


|detzU+l  ,k+l ) |  =  | detD [  =  0p(T“(v+1 )/2) , 
|detiU,k)|  =  |  detE  |  =  0p(T"v/2). 


v 
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Therefore 


lim  lim  Pr{ | Num|  <  cT"^2}  =  1 . 

G-Xx>  T-Xx, 


This  completes  the  proof.  □ 

Note  that  when  k  >  p  or  £  >  q  the  asymptotic  distribution  of  the  numerator 

of  r(k+l ,£+l )  involves  a  ratio  of  normal rrandom- variables .  Thus  we  expect 

to  see  a  Cauchy-like  behavior  for  the  numerator  of  r(k+l ,£+l ) . 

1/2 

When  k  =  p  or  £  =  q,  T  '  r(k+l,£+l)  has  an  asymptotic  normal  distribution 
with  mean  zero.  The  following  result  unifies  and  generalizes  the  well  known 
results  on  autocorrelation  coefficients  of  MA(q)  processes  and  on  partial 
autocorrelation  coefficients  of  AR(p)  processes. 

Theorem  3.2.  Consider  a  nondegenerate  ARMA(p,q)  process:  3(L)x.  =  a(L)v 
Let  Yj.  =  e(L)x^.  =  a(L)v^  and  let  p.  be  the  autocorrelation  coefficients  of 
Yf  For  such  that  k=P  £=q*T1/2r(k+l ,£+l)  are  asymptotical ly  jointly 

normally  distributed  with  mean  zero.  The  asymptotic  covariance  between 
T1/2r(k+l,£+l)  and  T1/2r(k'+l ,*'+1 )  is  given  by 


Y  Y 

pipi-(k-k ')-(£-£ 1 ) . 


(3.1) 


For  the  proof  the  argument  in  Section  5.6  of  Anderson  (1971)  can  be 
generalized  in  a  straightforward  way: 

Proof.  The  denominator  of  r(k+l,£+l)  converges  in  probability  to  VAR(Y)  = 

2  2 

VAR(Z)  =  cv£j_Qaj  for  all  k,  £  such  that  k  =  p  or  £  =  q.  Hence  we  only  need 
to  consider  the  numerator. 
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Let 

r,/Vr(1/T>j/tWi 

"^/T)  jiS(L)XtB(P)xt-k-!l-l 

T 

-(1/T)  l  cc(L)vte(P)xt_k_£.r 
t  1 

The  right  hand  side  is  the  same  as  the  numerator  of  r(k+l,£+l)  except  for  the 

A  A  A 

adjustment  of  endpoints  and  that  >$(»...  *3^)  are  replaced  by  3^»...»3p 

( ,0, . . . ,0) .  h.  corresponds  to  h .  in  formula  (27),  Section  5.6  of  Anderson 

K  J 

(1971).  With  the  same  argument  as  in  Anderson  (1971)  hk  ^'s  can  be  shown  to 
have  an  asymptotic  joint  normal  distribution  with  mean  zero.  Their  asymptotic 
covariances  are  given  by  1  i m-pE ( h^  ^h^,  ^ , ) .  We  evaluate  this  expected  value 
first.  Consider 

E ( YtZt-k-jt-l YsZs-k'-£'-l  ^ 

-E{a(L)vts(P)xt.k.1.1  <»(L)»ss(P)xs.k..t,.1}. 

Note  that  the  only  nonzero  contribution  to  the  expectation  comes  from  the  term 
where  the  same  v.-'s  are  taken  from  both  a(L)vt  and  a(L)vg.  This  term  contributes 

aiai-t+sE(vt-i)E(B(P)xt-k-£-l3(P)xs-k,-ji,-l) 

to  the  expectation.  Let  yt  =  E(YtYt_. )  =  I  ajaj+|-j|  denote  the 

autocorvariances  of  Y^.  By  the  time  reversibility  of  the  process  the  second 

Y 

expectation  is  the  same  as  Yt_k_il_s+|< i+£>  =  E(3(L)xt_k_£g(L)xs_kl _£l ) . 
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Now  summing  the  terms  involving  v.  .'s  we  obtain 

X*  “  1 

E(Ytzt-k-£-l  Vs-k'-a'-l)  =  YlsYlk-£-s+k'+£'  • 


Hence 


E(h 


k,£hk'  ,£ 


)  = 


T 


O/T)  l 

t,s=l 


Y  Y 

Yt-sYt-k-£-s+k'n ' 


Y  Y 

YiYi - ( k-k 1 )-($,-£ 1 )  ' 


Note  that  the  last  summation  is  only  finite.  Now  using  VAR(Yt)  =  VAR(Zt)  = 

Y  Y  Y  Y 

Y0  and  p..  =  Y-j/Yg  we  have 


lim  E(h 
T 


k^k'  ,£ 


)/[VAR(Y)VAR(Z)]  =  l 

i 


Y  Y 

p ip i - (k-k ' )-{z-z' ) 


and  this  gives  the  asymptotic  covariance. 

It  remains  to  verify  that  the  difference  between  h.  and  T1/2  x  (numerator 

K  j  36 

of  r(k+l,£+l)  converges  to  zero  in  probability.  Let  h.  denote  T1//2  times 

K  9  Z 

the  numerator  of  r(k+l  ,£+l ).  Then  (except  for  the  adjustments  of  end  points)  we 
have 


=  rl/2 


k,,.-hk,* =  T  1 j.0(BiBr6iBj)T1l1xt-ixt-k-ii-i+j 


■1/2 


i  J=0T  )3jj  a-j  ,k+£+l-j 


i  J=0  SiT  1  ai  ,k+£+l-j 


(3.2) 
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where  x^.j.  Now  Bj  »  Sj  and  (l/TJa^^.j  +  o(k+t+l-J-1) 

In  probability.  Considering  i  =  k,  j  =  p  in  the  first  term  and  i  =  p,  j  =  k 
in  the  second  term  on  the  extreme  right  hand  side  and  noting  k+£+l-(k+p)  = 
£+1 -p  >_  q+l-p  we  see  that  indices  fall  in  the  range  where  the  Yule-Walker 
equations  hold.  Hence  we  have 


l  h  T  ai,k+£+l-j  ^  °*  1  ' 

J 

l  6i  T  ai ,k+£+l-j  °*  J  = 

in  probability.  Therefore  the  two  terms  on  the  extreme  right  hand  side  of 

(3.2)  converge  to  zero  in  probability  and  this  completes  the  proof.  □ 

Y 

For  a  nondegenerate  ARMA(p,q)  process  y1-=Cov(Yt,Yt_1.)  can  be  consistently 
estimated  by 

aW  a  a  a 

y!  =  jj’jO.P+DjJ 

=  c(i)  -  a(q+T,p)’s(q,p)"’cr(1-l,-p) 

'U  Oj  f\j  /  o  ^ 

-  a(i+l ,p)'2(qsp) 1-1  a(q+l ,p) 

+  ^(q+1  ,p)l^(q»p)"1  &(i >p)2(q ,p) ,-1a(q+l ,p) . 

Let  rT  =  y^/yq,  i  =  1 , . . .  ,q.  Then  as  a  corollary  to  Theorem  3.2  we  have 

Corollary  3.1 .  Suppose  that  x^  is  a  nondegenerate  ARMA(p,q)  process.  Then 

T^2r(p+1  ,q+l )/[l+2  ^  (r  T)2]^2  has  an  asymptoti c  standard  normal  distribution. 

i=l  1 

Corollary  3.1  can  be  conveniently  used  as  a  test  statistic  for  the  null 
hypothesis  that  an  observed  process  is  a  nondegenerate  ARMA(p,q)  process. 
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4.  Recursive  Formulae. 

One  advantage  of  our  generalized  autocorrelation  function  is  that  it 
can  be  quickly  computed  by  recursive  formulae  presented  below.  It  is  a 
straightforward  generalization  of  the  well  known  recursive  procedure  for 
partial  autocorrelation  coefficients  (Durbin  (1960),  Levinson  (1946)).  Starting 
from  (q+l)-st  autocorrelation  coefficient  pq+,= p(1 ,q+l ) ,  the  formula  gives 
a  recursive  relation  for  obtaining  p(p+l,q+l)  for  successive  values  of  p.  For 
q=0  it  reduces  to  Durbin-Levinson  procedure. 

In  this  section  the  distinction  between  population  quantities  and  sample 
quantities  is  not  important.  Hence  we  omit  A  although  the  recursive  formulae 
are  used  with  sample  quantities  in  practice.  Also  we  use  p,q  as  general  running 
indices  rather  than  true  orders  of  AR  and  MA  parts. 

Let  q  ^  0  be  fixed  in  the  following  discussion.  We  define  p  dimensional 
vectors  b(p),  c(p),  d(i;p),  e(i;p)  (i=0,...,q)  as  follows: 

Definition: 

2(q,p) 'b(p)  =  -a(q+l,p) 

£(q,p)'c(p)  =  -a(q-p,p) 
d(i;p)  =  E(i,p)b(p), 
e(i  ;p)  =  E(i  ,p)c(p). 

Using  these  definitions,  y.  in  (3.3)  can  be  expressed  as 
y.  =  a(i)+b(p)'cf(i-l,-p)+a(i+l,p)'b(p) 

I  U  Mj  *\j 

+  d(i;p)'b(p),  i  =  0,...,q. 


(4.1) 

i  —  0, . . .  ,q , 
i  =  0, . . .  ,q. 
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Then  from  (2.2)  we  have 

p(p+l,q+l)  =  [a(p+q+l)+a(p+q,-p)'b(p)]/Y0  . 

Now  we  give  the  initial  condition  and  updating  formulae  for  b(p) ,  c(p) , 
d(i;p),  e(i;p). 

Initial  condition. 

0  =  b(0)  =  c(0)  =  jj(i;0)  =  e(i;0), 
i  =  0,. . .  ,q. 


Actually  these  are  zero-dimensional  and  the  condition  is  trivial. 
Updating.  Let 


b(p+l)  = 


b(1)(p+l)\ 

/  d^(i  ;p+l) 

,  d(i ;p+l )  -  [ 

Vi(p+1)  / 

\  Vi(i;p+1) 

dp+1(i;p+l) 

are  scalars  and  b^(p+l),  d^( 

‘X/  Oj 

i  =  0 


are 


p  dimensional  vectors.  Let 


c(p+l) 

r\j 


/c-|(p+l)  \ 

\«<2>(p+1V 


e(i ;p+l) 

'Ll 


;p+D 
(i  ;p+l) 


where  c^  (p+1 ) ,  e^  (i  ;p+l )  are  scalars  and  (p+1 )  and  ^ ^ 2 ^  ( i  ;p+l )  are  p 
dimensional  vectors.  Then 


f  cr(q+p+l )  +  ^(q+p,-p)  'b(p) 

bp+l<P+1)  =  ‘  o(q)  +  a(q+p,-p)'c(p) 

‘Xi  Of 

b(1)(p+l)  =  b(p)  +  bp+1(p+l)c(p)  , 
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a(q-p-l)  +  £(q-1,-p)'£(p) 
o(qj  +  £(q-l ,-p)  'j^(p)  ’ 

$(p)  +  c1  (p+l)Jg(p) , 

dp+l(i,p+1)  =  t)p+1  (p+1  )cr(i )  +  a(i-p,p)'b^1^(p+l)J 

^^(1;p+l)  =  jJ(i;p)  +  bp+1(p+l)  [j=(i;p)  +  a(i+p,-p)], 

e7  ( i  »P+1)  =  c-|(p+l)a(i)  +  a(i+l,p)'  £^(p+l), 

(2) 

S  (i;p+l)  =  $(i ;p)  +  c-j(p+l)  [^(i;p)  +  jj(i-l  ,-p)]. 

These  formulae  can  be  readily  verified  by  writing  (4.1)  for  p+1  in 
appropriately  partitioned  form.  When  q  =  0  <:(p)  equals  b(p)  with  its  elements 
in  the  reversed  order.  <i(0;p)  reduces  to  -a(l,p)  and  £(0;p)  reduces  to  -a(l,p). 
Hence  these  need  not  be  calculated  separately.  However  for  general  q  >  0, 

these  quantities  form  essential  working  values  and  have  to  be  stored  in  memory 
for  each  updating  step. 


c-j  (p+1 )  =  - 
S(2)(p+1)  - 


5.  Simulation  Results. 

Here  we  present  some  simulation  results.  An  ARMA (1,1)  process 
(3(L)  =  1-.5L,  a(L)  =  1+L)  and  ARMA  (3,3)  process  (g(L)  =  (1-.5L) (1+.5L2) , 
“(L)  =  (1+.5L) (1+L  ))  of  length  200  were  generated  100  times.  For  each  pair 
(p,q)  the  null  hypothesis  HQ:  p(p+l,  q+1 )  =0  was  tested  using  Corollary  3.1 
with  asymptotic  significance  level  .1.  The  entries  of  the  following  tables 
show  how  many  times  the  null  hypothesis  was  rejected  out  of  100  trials.  In 
Table  1  the  asymptotic  expected  number  of  (2,2)  element  is  10  (in  comparison 
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to  8  which  was  actually  observed).  In  Table  2  the  same  holds  for  (4,4) 
element.  The  pattern  implied  by  the  property  (P4)  of  p ( k , Jt)  in  Section 
be  clearly  seen  in  these  tables. 


p\q 

100 

100 

100 

99 

80 

65 

50 

37 


Table  1 


xt  -  -5xt-i  =  vt  +  Vl 

number  of  rejections  out  of  100  trials 


100 

8 

6 

5 

12 

6 
9 
3 


49 

8 

6 

4 

2 

4 

2 

2 


19  13  12 
3  5  6 
3  2  2 
3  2  0 
0  0  1 
0  1  0 
1  1  1 
2  0  0 


10 

4 

0 

1 

1 

1 

0 

0 


9 

7 

0 

1 

0 

1 

0 

0 


Table 

2 

xt"5xt-i+' 

>5xt-2“ 

25x  = 

•oxt-3 

V‘5vt 

~l+vt-2+"5v 

t-3 

1 

number 

of  rejections 

out  of 

100  trials 

1 

100 

100 

58 

12 

n 

9 

6 

8 

99 

87 

37 

8 

13 

11 

5 

7 

36 

10 

65 

7 

14 

2 

2 

0 

11 

54 

0 

8 

0 

0 

3 

1 

79 

3 

1 

2 

0 

1 

1 

0 

26 

41 

0 

4 

0 

0 

0 

0 

52 

3 

5 

1 

1 

1 

0 

1 

11 

34 

2 

4 

1 

0 

0 

0 

can 
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Appendix. 

Here  we  give  a  proof  of  Proposition  2.1.  We  consider  several  cases. 

Case  1 :  k  =  p . 

For  the  case  k  -  p,  £  =  q,  Anderson  (1971,  p.240)  shows  that  z(q,p)'  is 

nonsingular.  For  the  case  £  >  q,  Anderson's  argument  can  be  modified  in  a 

straightforward  way  to  show  that  z(£,p)'  is  nonsingular.  The  essential  condition 

for  the  proof  of  this  case  is  that  s  t  0. 

P 

Case  2:  k  >  p, £  =  q. 


We  shall  show  that  z(q,k)' 
condition  for  this  case  will  be 
First  we  show  that  z(q,p+l)'  is 
want  to  show  that  this  implies  £  =  0. 
written  as 


The  essential 
We  argue  by  induction. 
Consider  z(q,p+l)'£,  =  j].  We 
In  the  partitioned  form  this  can  be 

(Al) 

From  the  second 


A(q)  a(q-l,-p)\  \ 

\«(q+1*p)  /  \£(2)/ 

where  £-j  is  a  scalar  and  £(2)  ls  a  p-dimensional  column  vector, 
set  of  equations  we  have 


is  nonsingular  in  this  case, 
the  asumption  b^O. 
nonsingular. 


5(2)  =  -5i£(q.p),“1$(q+1,p)  = 


(A2) 


where  =  (B-j , . . .  ,6  ) 1 .  Then  the  first  equation  recudes  to 
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P 

0  =  Ci  (  l  a(q-i)e-). 
1  i=0  1 


But 


a(q-i)Bi  =  E{[6(L)xt]xt_q} 
*  E{[c(L)vt]xt.q> 


aqE(W*V  = 


2 

aqV 


Hence  0  =  s-jo  a^.  This  implies  c-j  =  0  and  consequently  ^2)  =  and  4  = 

This  proves  that  z (q ,p+l ) '  is  nonsingular. 

For  induction  suppose  that  z(q,j)'  is  nonsingular.  We  want  to  show  that 
z(q,j+l)'is  nonsingular  as  well.  The  argument  is  the  same  as  for  the  case 
j  =  p.  The  differences  are  that  p  is  replaced  by  j  in  (A1 )  and  £/-j\  is 
replaced  by  (g^ , . . . 9gpS0,. . . , ) 1  in(A2). 

This  proves  that  |z(q,k)'|  f  0  for  all  k  >  p. 

Case  3.  k  >  p,  £>  q,  and  k  -  p  £  £  -  q. 

We  shall  show  that  v(£(£,k) ')  =  k-p  and  ^  , . . . ,tjk_p  given  in  Proposition 
2.1  form  a  basis  of  N(z(£,k)'). 

'X/ 

The  upper  left  pxp  corner  of  z(£,k)'.  is  z(£,p)‘.  As  we  have  already 
shown  z  ( £ ,  p ) 1  is  nonsingular.  Therefore  rank  (z(£,k)')  £  p  or  v(z(£,k)')  £  k-p. 
Now  we  want  to  show  that  v(z(£,k) 1 )  £  k-p.  In  order  to  show  this  it  suffices 
to  check  that  > . . .  »r^k_p  given  in  Proposition  2.1  belong  to  N(z(£,k) 1 )  because 
they  are  clearly  linearly  independent  and  then  v(z(£,k) 1 )  =  dim  N(z(£,k) 1 )  £ 

k-p.  Note  that  in  this  case  ^k_p  =  (0 . 0,gQ,. . .  ,g  )  and  the  element  at  the 

upper  right  corner  of  z(£»k)'  i  s  o{  £-k+l ) .  Now  k-p  ££  -  q  implies  £-k+l  £  q+l-p. 
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This  implies  that  indices  fall  in  the  range  where  the  Yule-Walker  equations 
hold.  Then  £U,k)  ,  i  =  1 .  ,k-p  simply  reduce  to  the  Yule-Walker 

equations  and  therefore  0  =  z^k)'^.,  i=l,...,k-p.  This  proves  that 
v(s(£,k)  )  -  k-p.  It  has  been  also  shown  that  »  •  •  •  >%_p  a  basis  of 
N(£U,k)'). 

Case  4.  k  >  p,  a  >  q,  and  k  -  p  >  £  -  q. 

Let  r  -  q-£+k.  Then  r  >  p.  The  element  at  the  upper  right  corner  of 
£(£,k)'  is  cr(£-k+l).  Now  (£-k+l )+r-l=q .  We  see  that  the  upper  right  r*r 
corner  of  z(£,k)'  is  s(q,r)‘.  We  have  already  shown  that  z(q,r)'  is 
nonsingular.  Therefore  rank  (z(£,k)')  >  rank  (z(q,r)')  =  r  or  v(z(£,k)')  < 
M-  Now  we  want  to  show  that  v(£(£,k)')  >  £-q.  As  in  the  previous  case 
it  suffices  to  show  that  tj-, > • • • belong  to  N(z(£,k)').  In  this  case 
%-q  has  additional  k-(v-l )- (p+1 )  =  k-p-(£-q)  zeros  at  the  end  and  the  last 
nonzero  element  of  is  the  (£-q+p)-th  ( :£-q+p=v-l+p+l )  element.  Therefore 
the  last  nonzero  term  in  o(s>,- k)'^_q  is  3pa(£-(£-q+p)+l )  =  epa(q+l-p).  We 
see  that  again  the  indices  fall  in  the  range  where  the  Yule  Walker  equations 
hold  and  we  obtain  0  =  z(£,k) 'ri^. ,  i  =  l,...,£-q.  This  implies  that  v(z(£,k)') 
£-q  and  ^  » •  •  •  _q  form  a  basis  of  N(l(£sk)'). 

All  cases  have  been  examined  and  this  completes  the  proof.  □ 
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